Custom LLM API timeout instead of default 10 minutes for self-hosted LLMs #4076

Belerafon · 2025-05-28T13:29:37Z

Related GitHub Issue

Closes: #3621

Description

Why: Roo Code often emits API socket-timeout errors when using self-hosted, slow LLMs — either while reading a large file or even during the initial prompt phase. Yet these slow models can be usable in autonomous scenarios for working on some private code source. It would be great to have a user-tunable setting to configure the API timeout for self-hosted providers (OpenAI-compatible, Ollama, LMStudio, etc.). The timeout can be up to 30 minutes to allow big file chunks or prompts to be fully processed. This way, even an older laptop could produce meaningful results overnight with roo code.

What This PR adds a timeout option to the settings for LMStudio, Ollama, and OpenAI-Сompatible providers, and forwards this value into the OpenAI class constructor. Translations for all supported languages have been included.

Test Procedure

Connected an OpenAI-compatible client to a local slow LLM and tested timeout settings of 5, 10, and 20 minutes.

Observed socket-timeout errors and retries occurring according to the configured timeout.

Repeated manual testing for LMStudio provider with the same timeout values.

Type of Change

🐛 Bug Fix: Non-breaking change that fixes an issue.
✨ New Feature: Non-breaking change that adds functionality.
💥 Breaking Change: Fix or feature that would cause existing functionality to not work as expected.
♻️ Refactor: Code change that neither fixes a bug nor adds a feature.
💅 Style: Changes that do not affect the meaning of the code (white-space, formatting, etc.).
📚 Documentation: Updates to documentation files.
⚙️ Build/CI: Changes to the build process or CI configuration.
🧹 Chore: Other changes that don't modify src or test files.

Pre-Submission Checklist

Screenshots / Videos

Before

After

Documentation Updates

The UI guide should include:

Timeout for API requests to provider (min 5 min). If no response is received within this period, the request is retried. Increase this value for slower models.

Additional Notes

This is my first PR ever. So... do what you must.
Also, I am not a frontend developer, these code changes were done by LLM with my revisions.

Get in Touch

Important

Introduces customizable API timeout settings for self-hosted LLMs, updates UI components for user input, and adds translations in multiple languages.

Behavior:
- Adds customizable API timeout setting for LMStudio, Ollama, and OpenAI-Compatible providers in provider-settings.ts.
- Applies timeout setting in lm-studio.ts, ollama.ts, and openai.ts by converting minutes to milliseconds.
UI Components:
- Updates LMStudio.tsx, Ollama.tsx, and OpenAICompatible.tsx to include input fields for API timeout settings.
Translations:
- Adds translations for API timeout settings in multiple language files, including settings.json for ca, de, en, es, fr, hi, id, it, ja, ko, nl, pl, pt-BR, ru, tr, vi, zh-CN, and zh-TW.

^{This description was created by}^{for 70e9ab6. You can customize this summary. It will automatically update as commits are pushed.}

ellipsis-dev · 2025-05-28T13:31:52Z

webview-ui/src/components/settings/providers/LMStudio.tsx

The label translation key here is settings:providers.openAiApiTimeout, but this is in the LMStudio component and the value comes from lmStudioApiTimeout. Consider renaming the translation key to be consistent (e.g., lmStudioApiTimeout).

Suggested change

<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>

<label className="block font-medium mb-1">{t("settings:providers.lmStudio.apiTimeout")}</label>

Since the LMStudio, Ollama, and OpenAI-Compatible providers all use the OpenAI API library, the timeout name, description, and behavior are identical. Therefore, a common translation key is intentionally used for all three providers.

ellipsis-dev · 2025-05-28T13:31:52Z

webview-ui/src/components/settings/providers/Ollama.tsx

The placeholder text is using the translation key settings:placeholders.numbers.maxTokens, which doesn't seem appropriate for a timeout field. Consider using a key that matches the timeout setting for consistency.

Suggested change

placeholder={t("settings:placeholders.numbers.maxTokens")}

placeholder={t("settings:providers.openAiApiTimeout")}

ellipsis-dev · 2025-05-28T13:31:53Z

webview-ui/src/components/settings/providers/Ollama.tsx

The label is using the translation key settings:providers.openAiApiTimeout, but this file is for Ollama. It looks like a copy-paste error. Consider updating this to a key that reflects Ollama (e.g., settings:providers.ollamaApiTimeout).

Suggested change

<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>

<label className="block font-medium mb-1">{t("settings:providers.ollamaApiTimeout")}</label>

ellipsis-dev · 2025-05-28T13:31:53Z

webview-ui/src/components/settings/providers/OpenAICompatible.tsx

Typo / copy-paste issue: The placeholder translation key in this field is set to settings:placeholders.numbers.maxTokens, which doesn't match the field's purpose (API timeout). Consider using a more appropriate translation key (e.g. one related to timeout) for clarity.

Suggested change

placeholder={t("settings:placeholders.numbers.maxTokens")}

placeholder={t("settings:placeholders.numbers.timeout")}

This seems like a valid feedback, would using openAiApiTimeout here make more sense?

daniel-lxs · 2025-05-28T16:46:17Z

Hey @Belerafon, can you take another look at the translations? It seems that you could use translations that are more relevant.

FrancoFun · 2025-05-28T18:39:43Z

src/api/providers/lmstudio.ts

Isn't the OpenAI class timeout in seconds though? https://github.com/openai/openai-python?tab=readme-ov-file#timeouts

Node.js OpenAI library’s TypeScript definitions (see screenshot), the explicitly states:

“The maximum amount of time (in milliseconds) that the client should wait for a response from the server before timing out a single request.”

Moreover, the built-in default is set here:

timeout: options.timeout ?? 600000 /* 10 minutes */
—so here it’s definitely milliseconds, not seconds. May be a python library in seconds...

Belerafon · 2025-05-28T20:18:22Z

Hey @Belerafon, can you take another look at the translations? It seems that you could use translations that are more relevant.

I’ve fixed the placeholder translation—it was clearly a bug. However, I’ve kept using
t("settings:providers.openAiApiTimeout") and
t("settings:providers.openAiApiTimeoutDescription")
for all three providers, since the texts are identical and they all rely on the same OpenAI library.

By the way, I’ve noticed an issue with the tests for my PR: while npm test passes locally on my Windows 11 host, the platform-unit-test (ubuntu-latest) job on GitHub is failing. I’m not entirely sure what’s going wrong and would appreciate any help.

daniel-lxs · 2025-05-28T21:03:23Z

Hey @Belerafon, can you try rebasing your branch against main?

Belerafon · 2025-05-29T07:17:59Z

Hey @Belerafon, can you try rebasing your branch against main?

Looks like I did it

FrancoFun · 2025-05-30T01:47:21Z

My guess is that you will also have to update the test. It probably fails because it doesn't expect a timeout parameter:

api/providers/__tests__/openai.spec.ts:101:30
it("should set default headers correctly", () => {
// Check that the OpenAI constructor was called with correct parame…

Adding:
timeout: expect.any(Number),
in openai.spec.ts will hopefully do the trick.

I don't see this specific test for Ollama and LMStudio, so they should be fine.

vercel · 2025-05-30T12:38:10Z

@Belerafon is attempting to deploy a commit to the Roo Code Team on Vercel.

A member of the Team first needs to authorize it.

Belerafon · 2025-05-30T12:51:37Z

Adding: timeout: expect.any(Number), in openai.spec.ts will hopefully do the trick.

Thanks for your help, now the test passes! except [Vercel] - Requires authorization to deploy. I have no idea why I would need to deploy something somewhere with my small commit, but I guess it's beyond my capabilities.

FrancoFun · 2025-05-30T17:44:23Z

My guess is that you need to remove the "Draft / In Progress" label and assign the "Needs Preliminary Review" label. You can also check the "updated tests have been added to cover my changes." in the checklist in your first post.

daniel-lxs · 2025-05-30T17:51:09Z

Hey @Belerafon you can ignore that vercel check, it's something we are setting up. Also you don't need to change the labels at all.

packages/types/src/provider-settings.ts

daniel-lxs · 2025-05-30T18:40:19Z

webview-ui/src/i18n/locales/hi/settings.json

It seems like the Hindi translation for openAiApiTimeoutDescription is still in English.

daniel-lxs · 2025-05-30T19:33:05Z

Hey @Belerafon, I noticed that the timeout description mentions "min 5 min", but the input field accepts any positive number.

Should we add validation to enforce the 5-minute minimum? Or perhaps use 10 minutes as the minimum since that's the OpenAI library's default timeout?

Belerafon · 2025-05-30T19:42:01Z

Hey @Belerafon, I noticed that the timeout description mentions "min 5 min", but the input field accepts any positive number.

Should we add validation to enforce the 5-minute minimum? Or perhaps use 10 minutes as the minimum since that's the OpenAI library's default timeout?

I actually discovered this limitation (minimum 5 minutes) with my manual local tests. I didn't find any official information about it. Maybe the best way would be to just remove this information (min 5 min) from the description. But someone could potentially report a bug trying to set 1 minute and it won't work.

daniel-lxs · 2025-05-30T20:24:13Z

@Belerafon
I understand what if we change the input for a slider, and set the min value to 10, and maybe the max value to 120 minutes or something like that?

What do you think?

Belerafon · 2025-05-30T20:32:12Z

@Belerafon I understand what if we change the input for a slider, and set the min value to 10, and maybe the max value to 120 minutes or something like that?

What do you think?

A shorter timeout is helpful for cloud providers that normally reply quickly but occasionally hang indefinitely and require a retry.
Discussion #1658 request a way to introduce such timeouts (if I understood correctly).
If timeout can potentially be under default 10 minutes, it seems better no to constrain users to a higher minimum.

120 minutes isn't the best maximum either for my taste - I might want to run the new DeepSeek-R1-0528 on my laptop tonight and get something special and wonderful in the morning, but not Repeat 5...

daniel-lxs · 2025-05-30T20:40:31Z

@Belerafon

I understand, just trying to set this up in a way where the user doesn't accidentally set the timeout to an invalid value.

Belerafon · 2025-05-30T21:03:11Z

@Belerafon

I understand, just trying to set this up in a way where the user doesn't accidentally set the timeout to an invalid value.

In normal operation with standard providers, users can set any timeout value from 1 minute to infinity without affecting RooCode's functionality. In problematic cases, this timeout flexibility can be beneficial.

Belerafon · 2025-06-06T14:17:03Z

@daniel-lxs

I think I have implemented all fixes we discussed.

Could you please let me know if there’s anything else that needs to be addressed so that I can mark this PR as “Ready for review” and request approval from the code owners?

Thank you.

Roo Code often emits API socket-timeout errors when using self-hosted, slow LLMs — either while reading a large file or even during the initial prompt phase. Yet these slow models can be usable in autonomous (e.g. overnight) scenarios for working on some private code source. It would be great to have a user-tunable setting to configure the API timeout for self-hosted providers (OpenAI-compatible, Ollama, LMStudio, etc.). The timeout can be up to 30 minutes to allow big file chunks or prompts to be fully processed. This way, even an older laptop could produce meaningful results overnight with roo code.

Update the test to verify that the OpenAI client is initialized with a timeout parameter, while maintaining the existing test structure and assertions for other configuration options.

ellipsis-dev · 2025-06-19T13:53:16Z

webview-ui/src/components/settings/providers/LMStudio.tsx

+					})(),
+				}}
+				className="w-full mt-4">
+				<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>


The translation key used in this label is "settings:providers.openAiApiTimeout", which seems inconsistent with the LMStudio component (e.g., other keys use "lmStudio"). This might be a typographical error—please check if the key should be corrected to match the LMStudio naming, such as "settings:providers.lmStudioApiTimeout".

^{This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.}

ellipsis-dev · 2025-06-19T13:53:17Z

webview-ui/src/components/settings/providers/Ollama.tsx

+					})(),
+				}}
+				className="w-full mt-4">
+				<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>


Typographical issue: The label text key here is "settings:providers.openAiApiTimeout" but given that this configuration is for Ollama (using ollamaApiTimeout), it seems like it might be a mistake. Please verify if this should be "settings:providers.ollamaApiTimeout" instead.

Suggested change

<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>

<label className="block font-medium mb-1">{t("settings:providers.ollamaApiTimeout")}</label>

^{This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.}

ellipsis-dev · 2025-06-19T13:53:17Z

webview-ui/src/components/settings/providers/Ollama.tsx

+				<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>
+			</VSCodeTextField>
+			<div className="text-sm text-vscode-descriptionForeground -mt-2 mb-2">
+				{t("settings:providers.openAiApiTimeoutDescription")}


Typographical issue: The description text key is "settings:providers.openAiApiTimeoutDescription" which may be unintended given the context of using Ollama. Confirm whether this key should refer to Ollama rather than OpenAI.

Suggested change

{t("settings:providers.openAiApiTimeoutDescription")}

{t("settings:providers.ollamaApiTimeoutDescription")}

^{This comment was generated because it violated a code review rule: irule_C0ez7Rji6ANcGkkX.}

hannesrudolph · 2025-07-07T20:03:08Z

stale

Belerafon requested review from cte and mrubens as code owners May 28, 2025 13:29

github-project-automation bot added this to Roo Code Roadmap and Roo Code Roadmap May 28, 2025

github-project-automation bot moved this to New in Roo Code Roadmap May 28, 2025

github-project-automation bot moved this to Triage in Roo Code Roadmap May 28, 2025

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. enhancement New feature or request labels May 28, 2025

ellipsis-dev bot reviewed May 28, 2025

View reviewed changes

daniel-lxs moved this from Triage to PR [Draft / In Progress] in Roo Code Roadmap May 28, 2025

hannesrudolph added the PR - Draft / In Progress label May 28, 2025

FrancoFun reviewed May 28, 2025

View reviewed changes

daniel-lxs added PR - Changes Requested and removed PR - Changes Requested labels May 28, 2025

Belerafon force-pushed the API_Request_Timeout branch from 46ce635 to 9827fd3 Compare May 29, 2025 07:09

dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels May 29, 2025

Belerafon force-pushed the API_Request_Timeout branch from 9827fd3 to 60c6926 Compare May 30, 2025 12:38

dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels May 30, 2025

daniel-lxs reviewed May 30, 2025

View reviewed changes

daniel-lxs added PR - Needs Review and removed PR - Needs Review labels May 30, 2025

Belerafon force-pushed the API_Request_Timeout branch from fc449a0 to ab23de9 Compare May 31, 2025 05:58

daniel-lxs marked this pull request as draft June 3, 2025 23:36

Belerafon and others added 7 commits June 19, 2025 16:06

Placeholder text bugfix

5b3964d

test(openai): add timeout check to OpenAI client initialization test

665f0c9

Update the test to verify that the OpenAI client is initialized with a timeout parameter, while maintaining the existing test structure and assertions for other configuration options.

refactor: remove describe

0e9c2fb

refactor: remove describe

6aa0163

refactor: remove describe

0bd84b7

fixed Hindi translation for openAiApiTimeoutDescription

f3237a6

Belerafon force-pushed the API_Request_Timeout branch from ab23de9 to f3237a6 Compare June 19, 2025 13:25

fixed Indonesian translation for openAiApiTimeoutDescription

70e9ab6

Belerafon marked this pull request as ready for review June 19, 2025 13:50

Belerafon requested a review from jr as a code owner June 19, 2025 13:50

ellipsis-dev bot reviewed Jun 19, 2025

View reviewed changes

daniel-lxs marked this pull request as draft June 19, 2025 16:50

hannesrudolph closed this Jul 7, 2025

github-project-automation bot moved this from PR [Draft / In Progress] to Done in Roo Code Roadmap Jul 7, 2025

github-project-automation bot moved this from New to Done in Roo Code Roadmap Jul 7, 2025

	<label className="block font-medium mb-1">{t("settings:providers.openAiApiTimeout")}</label>
	<label className="block font-medium mb-1">{t("settings:providers.lmStudio.apiTimeout")}</label>

	placeholder={t("settings:placeholders.numbers.maxTokens")}
	placeholder={t("settings:providers.openAiApiTimeout")}

	{t("settings:providers.openAiApiTimeoutDescription")}
	{t("settings:providers.ollamaApiTimeoutDescription")}

Custom LLM API timeout instead of default 10 minutes for self-hosted LLMs #4076

Custom LLM API timeout instead of default 10 minutes for self-hosted LLMs #4076

Uh oh!

Conversation

Belerafon commented May 28, 2025 • edited by ellipsis-dev bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Related GitHub Issue

Description

Test Procedure

Type of Change

Pre-Submission Checklist

Screenshots / Videos

Documentation Updates

Additional Notes

Get in Touch

Uh oh!

ellipsis-dev bot May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Belerafon May 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot May 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot May 28, 2025

Choose a reason for hiding this comment

Uh oh!

ellipsis-dev bot May 28, 2025

Choose a reason for hiding this comment

Uh oh!

daniel-lxs May 28, 2025

Choose a reason for hiding this comment

Uh oh!

daniel-lxs commented May 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FrancoFun May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Belerafon May 28, 2025

Choose a reason for hiding this comment

Uh oh!

Belerafon commented May 28, 2025

Uh oh!

daniel-lxs commented May 28, 2025

Uh oh!

Belerafon commented May 29, 2025

Uh oh!

FrancoFun commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vercel bot commented May 30, 2025

Uh oh!

Belerafon commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

FrancoFun commented May 30, 2025

Uh oh!

daniel-lxs commented May 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

daniel-lxs May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

daniel-lxs commented May 30, 2025

Uh oh!

Belerafon commented May 30, 2025

Uh oh!

daniel-lxs commented May 30, 2025

Uh oh!

Belerafon commented May 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

daniel-lxs commented May 30, 2025

Uh oh!

Belerafon commented May 30, 2025

Belerafon commented May 28, 2025 •

edited by ellipsis-dev bot

Loading

daniel-lxs commented May 28, 2025 •

edited

Loading

FrancoFun commented May 30, 2025 •

edited

Loading

Belerafon commented May 30, 2025 •

edited

Loading

daniel-lxs May 30, 2025 •

edited

Loading

Belerafon commented May 30, 2025 •

edited

Loading